Bayesian Nonparametrics is a class of models with a potentially infinite number of parameters. High flexibility and expressive power of this approach enables better data modelling compared to parametric methods. Bayesian Nonparametrics is used in problems where a dimension of interest grows with data, for example, in problems where the number of features is not fixed but allowed to vary as we observe more data. Another example is clustering where the number of clusters is automatically inferred from data. The Statsbot team asked a data scientist, Vadim Smolyakov, to introduce us to Bayesian Nonparametric models.

There is no shortage in tutorials and beginner training for data science. Most of them focus on "report" data science. A one-time activity is needed to dig into a data set, clean it, process it, optimize hyperparameters, call .fit() On the other hand, SaaS or mobile apps are never finished, always changing and upgrading complex sets of algorithms, data, and configuration. Machine learning often expands functionality of existing applications -- recommendations on a web shop, utterances classification in a chat bot, etc.

A well-calibrated estimator for the conditional probabilities should obey this equation. Once we have derived a statistical classifier, we need to validate it on some test data. This data should be different from that used to train the classifier, otherwise skill scores will be unduly optimistic. This is known as cross-validation. The confusion matrix expresses everything about the accuracy of a discrete classifier over a given database and you can use it to compose any possible skill score. Here, we are going to cover two that are rarely seen in the literature, but are nonetheless important for reasons that will become clear.

A well-calibrated estimator for the conditional probabilities should obey this equation. Once we have derived a statistical classifier, we need to validate it on some test data. This data should be different from that used to train the classifier, otherwise skill scores will be unduly optimistic. This is known as cross-validation. The confusion matrix expresses everything about the accuracy of a discrete classifier over a given database and you can use it to compose any possible skill score. Here, we are going to cover two that are rarely seen in the literature, but are nonetheless important for reasons that will become clear.

If you have used machine learning to perform classification, you might have heard about Support Vector Machines (SVM). Introduced a little more than 50 years ago, they have evolved over time and have also been adapted to various other problems like regression, outlier analysis, and ranking. SVMs are a favorite tool in the arsenal of many machine learning practitioners. At [24]7, we too use them to solve a variety of problems. In this post, we will try to gain a high-level understanding of how SVMs work.

You may know it's impossible to define the best text classifier. Unlike that, text classification is still far from convergence on some narrow area. Along with the high-level discussion, we offer a collection of hands-on tutorials and tools that can help with building your own models. The toolbox of a modern machine learning practitioner who focuses on text mining spans from TF-IDF features and Linear SVMs, to word embeddings (word2vec) and attention-based neural architectures. When researchers compare the text classification algorithms, they use them as they are, probably augmented with a few tricks, on well-known datasets that allow them to compare their results with many other attempts on the same problem.

Data scientists are one of the most hirable specialists today, but it's not so easy to enter this profession without a "Projects" field in your resume. You need experience to get the job, and you need the job to get the experience. Seems like a vicious circle, right? Statsbot's data scientist Denis Semenenko wrote this article to help everyone with making the first simple, but yet illustrative data science projects which can take less than a week of work time. This means that you need to formulate the problem, design the solution, find the data, master the technology, build a machine learning model, evaluate the quality, and maybe wrap it into a simple UI.